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Whether  by  virtue  of  being  prepared  in  a  slowly  relaxing,  high-free  energy  initial  condition,  or  because 
they  are  constantly  dissipating  energy  absorbed  from  a  strong  external  drive,  many  systems  subject  to 
thermal  fluctuations  are  not  expected  to  behave  in  the  way  they  would  at  thermal  equilibrium.  Rather,  the 
probability  of  finding  such  a  system  in  a  given  microscopic  arrangement  may  deviate  strongly  from  the 
Boltzmann  distribution,  raising  the  question  of  whether  thermodynamics  still  has  anything  to  tell  us  about 
which  arrangements  are  the  most  likely  to  be  observed.  In  this  work,  we  build  on  past  results  governing 
nonequilibrium  thermodynamics  and  define  a  generalized  Helmholtz  free  energy  that  exactly  delineates  the 
various  factors  that  quantitatively  contribute  to  the  relative  probabilities  of  different  outcomes  in  far-from- 
equilibrium  stochastic  dynamics.  By  applying  this  expression  to  the  analysis  of  two  examples — namely,  a 
particle  hopping  in  an  oscillating  energy  landscape  and  a  population  composed  of  two  types  of 
exponentially  growing  self-replicators — we  illustrate  a  simple  relationship  between  outcome-likelihood 
and  dissipative  history.  In  closing,  we  discuss  the  possible  relevance  of  such  a  thermodynamic  principle 
for  our  understanding  of  self-organization  in  complex  systems,  paying  particular  attention  to  a  possible 
analogy  to  the  way  evolutionary  adaptations  emerge  in  living  things. 
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I.  INTRODUCTION 

It  has  long  been  understood  that  nonequilibrium  driving 
can  bring  about  novel  patterns  of  complex  organization  in 
thermally  fluctuating  many -body  systems.  Whether  in  fluid 
dynamics  [1,2]  or  so-called  “active  matter”  mixtures  [3,4], 
numerous  examples  have  been  characterized  where  quali¬ 
tatively  new  types  of  collective  behavior  become  possible 
because  a  large  set  of  interacting  degrees  of  freedom  are 
allowed  to  act  together  as  a  conduit  for  chemical  and  heat 
dissipation  in  surrounding  reservoirs. 

Many  have  hoped  that  statistical  thermodynamics  might 
provide  us  with  general  predictive  principles  for  the 
behavior  of  such  systems,  much  in  the  same  way  as  in 
the  equilibrium  case  where  the  Boltzmann  distribution 
applies.  However,  historically,  efforts  along  these  lines 
have  encountered  serious  difficulty  outside  of  the  near¬ 
equilibrium,  linear-response  regime  [5]  because  the  number 
of  time,  length,  and  energy  scales  that  are  important  in 
determining  the  dynamics  can  multiply  confoundingly 
once  external  drives  become  arbitrarily  strong. 

Nonetheless,  there  is  renewed  reason  to  anticipate  that  a 
thermodynamic  universality  might  yet  be  recognized  in 
strongly  driven  systems  that  exhibit  complex  collective 
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behavior.  In  the  last  two  decades,  many  fundamental  results 
in  nonequilibrium  statistical  mechanics  have  been  derived 
[6-15],  for  the  first  time  establishing  general  relationships 
between  far-from-equilibrium  probability  distributions  over 
states  and  the  thermodynamic  fluxes  that  accompany  their 
dynamical  evolution.  The  common  theme  in  these  findings 
is  that  time-reversal  symmetry  and  energy  conservation 
together  imply  that  the  rates  of  stochastic  transitions  in 
driven  open  systems  have  quantitative  relationships  with 
amounts  of  energy  and  matter  exchanged  with  surrounding 
baths.  In  particular,  it  has  been  shown  that  transitions  that 
are  more  statistically  irreversible  must  be  accompanied  by 
more  positive  amounts  of  total  entropy  production. 

We  aim  here  to  suggest  a  new  interpretation  for  one  of 
these  results  that  has  potential  relevance  to  our  under¬ 
standing  of  the  organization  of  driven  many-body  systems. 
In  particular,  we  point  out  that  the  likelihood  of  observing  a 
given  structure  to  emerge  in  nonequilibrium  evolution  is 
strongly  influenced  by  the  amount  of  absorption  and 
dissipation  of  work  during  its  history  of  formation.  We 
examine  the  mechanism  of  this  general  relationship  in 
simple  analytical  examples.  Subsequently,  taking  inspira¬ 
tion  from  the  way  evolutionary  adaptation  is  understood  in 
a  biological  context,  we  argue  that  many  structures  formed 
far  from  equilibrium  may  appear  to  have  been  specially 
selected  for  physical  properties  connected  to  their  ability  to 
absorb  work  from  the  particular  driving  environment,  and 
we  discuss  the  relevance  of  this  hypothesis  to  studying  the 
physics  of  self-organization. 
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II.  DISSIPATION  AND  DRIFT  IN  A 
TIME-VARYING  FIELD 

At  thermal  equilibrium,  it  is  well  known  that  the  like¬ 
lihood  of  observing  a  system  in  one  of  its  microstates  has  a 
simple  exponential  relationship  to  the  energy  of  that 
microstate.  Once  external  drives  push  the  system  away 
from  equilibrium,  however,  the  relationship  between  prob¬ 
ability  and  the  flow  of  energy  in  the  system,  in  general, 
becomes  more  complicated.  In  order  to  start  to  see  how,  let 
us  first  consider  a  simple  model  of  a  single  particle 
stochastically  hopping  in  a  landscape  of  discrete  energy 
levels.  Suppose,  for  such  a  system,  that  state  x,-  has  a 
particular  energy  En  and  each  pair  of  states  x,  and  x;  are 
separated  by  an  activation  barrier  of  energy  /!,,= 
Bji  >  ma x(Ej,  Ej ).  To  couple  to  a  thermal  bath  of  temper¬ 
ature  T  =  1//?  in  a  way  that  is  consistent  with  our  under¬ 
lying  assumptions  about  time-reversibility,  we  make  the 
Arrhenius  assumption  that,  when  our  particle  is  in  state  x(, 
it  always  has  a  constant  probability  rate  of  stochas¬ 
tically  hopping  to  state  x,-,  which  is  given  by  [16] 

r,w  =  r°.  exp  [-/?(£, 7  -  Ej)].  (1) 

Here,  r{\-  =  r®  is  a  rate  constant  specific  to  the  pair  of  states 
that  remains  unaffected  when  we  change  En  Ej,  or  Bir 
By  assuming  transition  probability  rates  to  take  this  form, 
we  are  guaranteed  that,  so  long  as  the  energies  of  states 
and  barriers  do  not  change  with  time,  the  probability 
distribution  for  the  location  of  our  hopping  particle  must, 
at  steady  state,  converge  on  a  Boltzmann  distribution 
ps(xj)  oc  exp[— fiEj]  that  will  obey  the  detailed  balance 
condition  ps{xi)/ Ps(xj)  =  rj^i/ri^j  required  by  time- 
reversal  symmetry. 

To  drive  the  system  away  from  equilibrium,  we  choose 
the  energies  £,(/)  and  to  be  arbitrary  functions  of 

time.  Work  is  done  on  the  system  whenever  the  energy  Ex¬ 
changes  while  the  particle  is  located  at  xh  and  heat 
A Q  =  Ej(t)  —  Ej ( t)  is  exhausted  into  the  surrounding 
thermal  bath  whenever  the  particle  hops  from  x,-  to  Xj  at 
time  t.  In  particular,  we  consider  an  energy  landscape 
composed  of  only  three  states  xx,  x2,  x3  arranged  in  a 
row  so  that  r°l2  =  r23  —  r  >  0  and  r°l3  —  0  (Fig.  1).  All 
three  states  are  initially  assumed  to  have  the  same  energy 
Ej  —  0  and  to  be  interlaced  with  two  barriers  of  equal 
height  Bl2  =  B2 3  =  A E  >  0. 

If  the  single  particle  is  placed  at  x2  at  t  =  0,  then,  in  the 
absence  of  any  external  drives,  it  is  obviously  equally 
likely  to  hop  to  xx  on  “the  left”  or  to  x3  on  “the  right”  in 
some  short  time  r  <s:  1/r  since  r2_>i  =  r2^3  =  re~^AE. 
Moreover,  the  probability  of  returning  to  x2  in  time  r  after 
making  that  first  jump  is  also  the  same  for  a  particle 
at  X,  or  x3.  We  may  then  choose  to  drive  the  system  at 
high  frequency  so  that  Ex{t)  —  —  A£'cos(a»f)/2  and 
B\2{t)  =  AE  —  AEcos(fflf)/2,  while  E2,  E2,  and  B2 3 
remain  constant  in  time  and  co  1  /t  r. 
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FIG.  1.  Three  separate  microstates  initially  of  equal  energy 
E  =  0  are  arranged  on  a  line  so  that  stochastic  transitions  may 
take  place  between  adjacent  locations  separated  by  activation 
barriers  of  height  A E.  When  state  xx  and  the  barrier  separating  it 
from  x2  are  driven,  such  that  their  energies  oscillate  in  time,  a 
preference  for  xx  over  x3  as  a  destination  for  finite  time  evolution 
from  x2  develops.  This  statistical  drift  is  necessarily  accompanied 
by  the  conversion  of  absorbed  work  into  dissipated  heat  in  the 
surroundings.  The  energies  of  the  driven  states  are  drawn  at  their 
extremal  values  in  blue,  with  dashed  lines  and  solid  lines, 
respectively,  in  phase  with  each  other. 

The  simple  point  here  is  that  by  contriving  the  external 
drive  in  the  manner  described,  we  have  necessarily  also 
introduced  a  strong  correlation  between  dissipation  and 
drift.  To  see  this,  we  note  that  the  rightward  hopping  rate 
r2^3  =  re~^AE  does  not  change  with  time.  In  contrast, 
the  leftward  hopping  rate  gets  an  extra  boost  from  the 
oscillation  of  the  barrier  energy  Bx2(t),  so  that 

r2^\(t)  =  rexp  [-/)(AE  -  A£’cos[o»r]/2)].  (2) 

So  long  as  (1AE :»  1,  leftward  hopping  events  are  much 
more  likely  to  happen  during  the  part  of  the  drive  cycle 
when  the  hopping  rate  is  maximal  because  of  the  expo¬ 
nential  rate  enhancement  that  comes  from  the  downswing 
of  B  12(f)-  At  this  maximum,  the  ratio  of  the  leftward  and 
rightward  rates  is 

~max 

—  exp  \flAE/2],  (3) 

^2-*3 

indicating  a  strong  bias  towards  landing  at  xx  rather  than 
at  x3  after  initially  being  placed  at  x2.  One  may  confirm 
this  bias  by  averaging  each  hopping  rate  over  a  whole 
drive  period  2 jt/co,  obtaining  r2^i  =  re~PAEIQ(/3AE/2)  > 
r2^3  =  re~PAE. 

At  the  same  time,  it  is  clear  this  bias  towards  one  outcome 
over  the  other  usually  requires  extra  dissipation  that  accom¬ 
panies  motion  in  the  likely  direction.  During  moments  in  the 
drive  cycle  when  transits  from  x2  to  xj  are  likely  (such  as  at 
t  =  0),  our  chosen  drive  has  guaranteed  that  E  will  be  at 
its  minimum  value,  so  a  hop  to  the  left  must  typically  be 
accompanied  by  a  positive  amount  of  dissipative  external 
entropy  production  /3(AQ)2^X  =  fiAE/2.  However,  hops  to 
the  right  will  not  dissipate  at  all  since  E2  =  E2  =  0  do  not 
change  with  time,  so  f3(AQ2 ^3)  =  0.  Thus,  we  can  see  that, 
at  least  in  this  case,  consistently  positive  entropy  production 
in  the  surroundings  and  concerted  drift  towards  a  more  likely 
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outcome  of  stochastic  evolution  are  two  sides  of  the 
same  coin. 

The  notion  that  the  flow  of  probability  density  in  driven 
systems  goes  in  directions  that  lead  to  more  dissipation  of 
heat  has  some  intuitive  appeal,  but,  not  surprisingly,  it  turns 
out  to  be  too  simple  to  be  true  in  general.  In  the  next 
section,  we  present  a  relationship  between  probability  flow 
and  thermodynamic  flux  that  does  apply  in  a  very  broad 
class  of  driven  systems,  thus  illustrating,  more  generally, 
what  role  dissipative  history  plays  in  determining  the 
outcome  of  a  nonequilibrium  dynamical  evolution. 

HI.  ENTROPY  PRODUCTION  AND 
STOCHASTIC  EVOLUTION 

The  general  scenario  of  interest  to  us  is  one  where  some 
large  number  of  interacting  particles  are  confined  to  a 
volume  V  that  is  held  in  contact  with  a  heat  bath  at  constant 
inverse  temperature  p=\/T.  In  accordance  with  the 
approach  pioneered  in  previous  works  [6],  the  classical 
Hamiltonian  of  the  whole  setup  is  written  as 

tftot  =  Hsys(x,A(t))  +  Hbath(  y)  +  /Iint(x,  y),  (4) 

where  x  =  {q[l\ p[l\ p^-}  accounts  for  all  the 
coordinate  degrees  of  freedom  for  the  N  particles  in  the 
system,  y  does  the  same  for  the  bath,  and  the  Hamiltonian 
functions  //sys,  //bath,  and  /iint  define  conservative  inter¬ 
actions  among  the  various  position  coordinates  of  system 
and  bath.  The  function  2(f)  plays  the  role  of  a  time-varying 
external  field  that  acts  exclusively  on  the  system  and  can  do 
work  on  the  coordinates  x.  Crucially,  hmi  is  assumed  to  be 
small,  so  the  term  merely  plays  the  formal  role  of  permit¬ 
ting  the  flow  of  energy  between  the  bath  and  the  system  and 
may  otherwise  be  ignored. 

The  standard  approach  [7]  to  modeling  such  a  system  is 
to  posit  that  the  coupling  to  the  heat  bath  introduces 
stochasticity  into  the  observed  dynamics,  so  if  one  starts 
off  at  some  particular  point  in  the  system’s  phase  space 
x(0)  at  time  t  =  0,  then,  for  a  given  choice  of  the  driving 
field  2(f),  there  is  some  probability  density  for  micro- 
trajectories  of  the  system  /rr[x(f)|x(0);2(f)]  that  expresses 
how  likely  one  would  be  to  observe  the  system  progressing 
through  a  given  series  of  subsequent  arrangements  x(f) 
over  time  t.  This  density  is  governed  by  the  microscopic 
reversibility  relation  put  forward  by  Crooks  [7]: 

JTt[xI(t-  rjlx^T)^?-  f)] 
*v[x(f)|x(0);2(f)] 

=  exp  (— /?A<2[x(f)]) 

—  exp  (-ASbath[x(f)]).  (5) 

The  above  equation  relates  the  probability  of  observing  a 
given  microtrajectory  to  the  probability  of  observing  the 
time-reversed  movie  trajectory  after  flipping  all  momentum 
coordinates  in  the  starting  state  [x^t)  =  (^/’(t), 


...)].  Because  of  the  underlying  time-reversal 
symmetry  of  Newton’s  laws,  the  ratio  of  the  forward  and 
reverse  probabilities  must  be  exactly  equal  to  the  expo¬ 
nential  of  the  heat  /?A<2[x(f)]  evolved  into  the  bath  in  the 
forward  direction  as  the  system  traverses  the  microtrajec¬ 
tory  x(f).  Since  this  heat  is  being  transferred  to  a  vast 
external  reservoir  at  constant  temperature  T  =  1  //?,  we  may 
accordingly  identify  it  thermodynamically  as  entropy 
production  and  write  /?A<2[x(f)]  =  ASbatb[x(f)].  Thus, 
the  Crooks  result  establishes  a  general,  exact,  microscopi¬ 
cally  detailed  relationship  between  statistical  irreversibility 
and  entropy  production  in  the  surrounding  bath,  and  for  the 
remainder  of  this  work,  we  treat  heat  evolution  and  external 
entropy  production  interchangeably. 

Recently  [14],  we  have  demonstrated  the  consequences 
of  the  above  relation  for  the  stochastic  dynamics  of 
arbitrarily  driven  nonequilibrium  macrostates.  We  may 
define  a  macrostate  by  labeling  an  arbitrary  collection  of 
microstates  as  sharing  some  macro-observable  property  I, 
and  in  that  case,  if  the  system  is  prepared  according  to  some 
controlled  experimental  procedure  to  be  in  I,  then  it  will 
implicitly  possess  some  nonequilibrium  density  over 
microstates  p(x|I)dx  =  pt(x)dx.  If  we  furthermore  des¬ 
ignate  some  other  disjoint  set  of  microstates  to  have 
macroproperty  II,  then  we  can  define  ;rr[I  ->  1 1; 2(f)] 
as  the  probability  that  the  system  is  observed  to  have 
property  1 1  after  stochastically  evolving  under  the  applied 
field  2(f)  for  time  r.  And  if  the  system  is  indeed  observed  to 
be  in  II,  then  it  will  implicitly  be  distributed  over  the 
microstates  available  to  it  according  to  some  new  density 
p(x|II,  I;2(f);r)  =  pf(x)dx.  In  this  case,  we  can  also 
define  the  probability  n)[\V  —>■  L’; 2(r  —  f)]  of  reverting 
back  to  I  if  particle  momenta  are  reversed  and  the  applied 
field  is  run  backward  (Fig.  2).  With  these  definitions  in 
hand,  it  may  be  shown  that 


FIG.  2.  An  arbitrary  nonequilibrium  macrostate  for  the  system 
of  interest  is  constructed  as  the  probability  density  pt(x)dx  over 
the  microstates  x  when  the  system  is  prepared  by  a  controlled 
procedure  to  have  some  macroscopic  property  I.  The  system  is 
externally  driven  for  a  period  of  duration  r  by  a  time- varying  field 
Hsys(x,  2(f))  while  in  contact  with  a  thermal  bath  of  inverse 
temperature  /?  that  absorbs  heat  A  Q.  Afterwards,  one  may  ask 
whether  the  system  now  has  a  new  macroscopic  property  II;  if 
so,  the  new  probability  distribution  over  microstates  is  implicitly 
given  by  pAx)dx. 
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n\[ IV  ->  It;2(r-f)] 
XT[ I  -  H;2(f)] 


(exp(  A^tot))  I_>II , 


(6) 


where  A5tot[x(f)]={ln  [/?;(x(0))]/[p/(x(t))]}+/?AG[x(0]- 

Put  succinctly,  this  result  says  that  statistical  irreversibility 
measured  on  arbitrary  macro-observables  still  has  an  exact 
quantitative  relationship  to  entropy  production,  with  two 
important  differences:  first,  that  one  must  account  for  the 
internal  entropy  difference  between  macrostates,  and  second, 
that  now  the  entropy  must  be  exponentially  averaged  in  the 
appropriate  way  over  the  statistical  weights  of  all  the  micro- 
trajectories  propagating  from  the  starting  macrostate  to  the 
ending  macrostate.  Of  course,  the  fundamental  physics  behind 
this  statement  remains  the  same  as  that  established  by  the 
microscopic  relation  in  Eq.  (5).  The  importance  of  layering  a 
coarse-graining  procedure  on  top  derives  only  from  the  fact 
that  experiments  on  many-body  systems  invariably  involve 
observing  or  measuring  a  limited  set  of  macroscopic  criteria  or 
quantities  in  the  system,  and  so  we  need  to  do  proper  book¬ 
keeping  on  the  number  of  microscopic  ways  that  the  system 
could  appear  the  same  to  an  experimenter  at  the  macro  level. 

Previously,  the  above  result  was  used  to  derive  an 
extension  of  the  second  law  of  thermodynamics,  which 
was  applied  to  studying  the  thermodynamic  constraints 
obeyed  by  self-replicators  as  they  grow.  Elere,  we  instead 
use  the  equation  to  give  a  thermodynamic  account  of  driven 
stochastic  evolution.  Thus,  let  us  consider  a  many-particle 
system  prepared  in  I  in  the  manner  described  above,  and  let 
us  suppose  that  we  have  defined  two  distinct  possible 
macroscopic  outcomes,  II  and  III,  that  could  lie  a  long 
way  in  the  system’s  future  (Fig.  3).  A  question  of  great 
interest  to  us  would  be  which  of  the  two  outcomes  should  be 
more  likely  given  Hsys(x,A(t)),  that  is,  given  the  way  the 
particles  inside  the  system  interact  with  each  other  and 
the  way  the  system  is  being  externally  driven.  Formally,  we 
may  write 


FIG.  3.  In  this  scenario,  we  consider  a  starting  nonequilibrium 
macrostate  I  that  is  driven  by  an  external  field  while  it  evolves 
stochastically  in  contact  with  a  thermal  bath.  We  may  ask 
whether,  after  some  finite  time  z,  it  is  more  likely  to  be  found 
in  one  of  two  other  possible  macrostates  II  and  III.  The  relative 
likelihood  of  these  two  different  possibilities  will  be  determined 
in  part  by  the  statistical  distributions  for  possible  values  of  the 
external  entropy  production  /?AQi^*  in  each  case. 


the  system  of  p,(x),  we  can  treat  any  correlation  between  W 
and  P/(x)  or  p,(x)  as  being  negligible  and  thus  write  out 
the  following  separate  terms: 
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Much  can  already  be  learned  from  the  setup  of  this  equation, 
but  a  few  additional  simplifying  assumptions  help  make 
its  structure  even  clearer:  If  we  assume  that  the  system  is 
driven  time  symmetrically  for  a  long  time,  we  can  dispense 
with  using  f  operators,  and  it  may  also  be  reasonable 
to  posit  that  there  is  no  correlation  between  x(0)  and 
x(r)  other  than  the  one  introduced  by  constraining  the 
starting  and  ending  points  to  lie  in  certain  macrostates. 
By  conservation  of  energy,  the  work  done  by  external 
fields  is  given  by  W[x(t)]  =  ()AQ[x(t)\  +  Hsys(x(z), 
2(t))  —  Hsys(x(0),  2(0)).  Since  W  depends  on  the  whole 
history  of  duration  r  for  the  driven  system  and  r  is  assumed 
to  be  much  longer  than  the  time  needed  to  lose  all  memory  in 


where  p*bz{x{t) ,  X{t))dx  =  lx&e  PH **(*(.*) A*)) dx/Ze^  t 
and  Z%t)t  =  fT&dxe-fiH’>-MMt».  This  result  follows 
because,  according  to  our  assumptions  and  definitions, 

Asjrxoi  =  ^  {[p?(x(°))]/[p/(x(t))]}  +  /w[x(f)]- 
M>L(x(°)>  m)z7mV[plM^  A(f))zI(ri,J}- 

The  first  term  in  Eq.  (8)  has  an  appealingly  intuitive 
meaning:  The  quantity  —In (pf/p*bz)*  <  0  is  a  functional 
over  the  nonequilibrium  distribution  in  macrostate  *  that  is 
only  equal  to  zero  when  Pf(x)  is  equal  to  the  Boltzmann 
distribution  pbz(x)  [which  can  be  shown  most  easily  for  a 
discrete  distribution  over  states  i  using  Lagrange  multi¬ 
pliers  by  solving  [d/dpf(i)]J2i(pf(i)2/pbz(i) -Xpf{i))  = 
0],  The  behavior  of  this  functional  for  small  variations 
from  pbz  has  been  examined  previously  in  this  context 
since  one  may  then  approximate  In  ^2jPf(i)2/Pbz(i)  = 
In  y~),p/-(i)eln(Tf/t|fe)  =  ln(eln(p//pfiz)j^  «  In  e(ln(P//p*z)>/  — 
£>/(*)  In  {[(P/(0]/[Pt>z(0]}’  the  last  of  which  is  equal  to 
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the  Kullback-Leibler  divergence  between  the  equilibrium 
and  nonequilibrium  distributions  [17].  Meanwhile,  F'Jr)  *  = 

— ^InZjjF  t  clearly  just  measures  the  equilibrium  free 

energy  that  would  be  obtained  if  the  system  were  equili¬ 
brated  at  inverse  temperature  />  while  being  constrained  to 
inhabit  the  states  belonging  in  macrostate  *.  Thus,  we  can 
recognize  that  the  quite  intuitive  effect  of  these  two  terms 
together  is  to  account  for  the  dynamical  tendency  of  any 
driven  system  to  inhabit  macrostates  of  low  equilibrium 
free  energy  and  to  progress  towards  thermal  equilibration 
within  those  macrostates.  Here,  we  follow  the  example 
established  in  previous  treatments  of  exponentially  aver¬ 
aged  work  distributions  [6]  and,  defining  the  so-called 
“dissipated  work”  for  a  trajectory  ending  in  *  as  W j  = 
W  —  Fe^s  f  +  j,  we  can  combine  the  second  and 
fourth  terms  in  the  above  equation  into  one  path  ensemble 
average  of 

Thus,  against  the  backdrop  of  an  expected  tendency  to 
thermally  equilibrate  dictated  by  the  first  term,  we  are  now 
in  the  position  to  weigh  the  effect  of  the  latter  two  terms 
remaining  in  the  equation.  The  first  of  these  reflects  the 
impact  of  kinetics.  Succinctly,  it  says  that,  in  a  given 
amount  of  time,  one  is  more  likely  to  propagate  to  a 
macrostate  from  which  one  is  more  likely  to  revert  back  in 
the  same  amount  of  time.  We  can  start  to  get  a  handle  on 
this  term  by  considering  a  simple  transition-state  theory 
model  of  a  one-step  chemical  reaction  between  initial  and 
final  states  [16]:  When  the  activation  free  energy  barrier  is 
lowered,  both  the  forward  and  reverse  rates  of  the  reaction 
are  accelerated.  Thus,  one  factor  we  would  expect  to  affect 
the  value  of  this  term  is  the  cumulative  impact  of  all  the 
kinetic  banners  that  need  to  be  traversed  in  order  to  get  from 
a  given  starting  state  to  a  given  ending  state  in  a  fixed 
amount  of  time.  Put  another  way,  one  more  rapidly  evolves 
to  states  that  are  closer  to  one’s  starting  point  and  separated 
from  it  by  fewer  high  barriers.  Indeed,  this  remains  the  case 
even  when  we  are  talking  about  ensembles  of  trajectories 
that  are  likely  in  the  forward  direction;  within  this  ensem¬ 
ble,  the  probabilities  of  trajectories  are  affected  by  the 
heights  of  barriers  that  must  be  traversed  whether  in  the 
forward  (likely)  or  reverse  (unlikely)  direction. 

In  light  of  this,  we  seem  to  be  presented  with  two 
significant  problems.  The  first  of  these  is  that  we  are 
purportedly  interested  in  knowing  how  much  more  likely 
we  are  to  see  1 1  happen  in  our  driven  system  than  III,  and 
yet  in  order  to  say  anything  concrete,  it  would  seem  that  we 
have  to  know  the  reversal  probabilities  n\,  which  requires 
complete  knowledge  of  all  the  rates  of  stochastic  transitions 
that  are  possible  in  our  system.  Put  another  way,  we  may  as 
well  run  a  fully  detailed  microscopic  simulation  and  just 
see  what  happens.  The  second  problem  is  the  well-known 
difficulty  of  averaging  exponentials  of  large  quantities: 
There  are  many  microtrajectories  that  run  from  I  to  II, 
and  in  general,  it  will  be  some  subensemble  of  highly 


improbable  ones  that  make  the  dominant  contribution  to 
(exp(—  if  we  average  over  all  possible  paths. 

This  puts  us  in  the  untenable  position  of  needing  to  know 
the  dissipation  associated  with  stochastic  events  in  our 
system  that  are  immeasurably  unlikely  to  occur  [18]. 

We  can  start  at  least  by  turning  two  problems  into  one  if 
we  note  that  everything  we  have  said  so  far  still  applies  if 
we  choose  not  to  average  over  all  paths  leading  from  one 
macrostate  to  another  but  rather  only  over  a  select  sub¬ 
ensemble  of  paths  representative  of  most  of  the  probability 
current.  Our  goal  is  to  make  an  inference  about  how  the 
probability  of  reaching  a  given  macrostate  is  related  to  the 
typical  thermodynamic  fluxes  that  are  associated  with 
getting  there.  Thus,  the  practical  meaning  of  this  restriction 
is  that  one  must  choose  a  quantitative  criterion  that  defines 
what  we  mean  by  “typical,”  which  should  generally  be 
determined  for  the  putposes  of  calculation  by  one’s  ability 
to  sample  the  distribution  of  entropy  production  in  the 
forward  direction.  Formally,  we  can  propose  a  procedure 
whereby  some  finite  number  J\f  »  1  of  paths  are  randomly 
drawn  from  the  forward  probability  distribution  for  paths 
leading  between  the  given  starting  and  ending  macrostates. 
Implicitly,  such  a  procedure  excludes  “abnormal”  types  of 
events  unlikely  to  occur  once  in  J\f  experiments.  Thus,  the 
associated  work  distribution  may  still  be  highly  non- 
Gaussian  and  have  substantial  contributions  to  (e~^Wd) 
from  higher-order  cumulants,  but  it  will  now  have  a  value 
that  is  determined  by  the  statistics  of  observable  events  of 
measurable  likelihood.  To  present  a  concrete  example,  if 
we  observed  the  same  seed  in  the  same  pot  of  soil  over  the 
course  of  several  weeks  in  a  million  repetitions  of  the  same 
experiment,  we  would  never  expect  to  see  a  fully  grown 
plant  spontaneously  self-assemble  in  the  last  ten  seconds  of 
the  interval  of  observation;  rather,  all  the  trajectories  we 
observed  (even  those  that  were,  in  some  sense,  “one  in  a 
million”)  would  look  like  a  seed  growing  gradually  into 
a  plant. 

If  we  restrict  the  average  to  these  “normal”  forward 
paths,  the  two  things  that  change  in  tandem  (since  we  are 
holding  the  starting  and  ending  points  of  the  ensemble 
of  trajectories  fixed)  are  the  dissipated  work  averages 
and  the  reversal  probabilities:  We  get  to  average  dissipation 
over  events  of  measurable  likelihood,  and  the  “price”  we 
pay  is  that  the  reversal  probabilities  now  correspond  only  to 
the  likelihoods  of  trajectories  that  look  like  reverse  movies 
of  trajectories  that  look  normal  in  the  forward  direction. 
This  method  works  because  excluding  unlikely  freak 
occurrences  from  our  ensemble  of  trajectories,  by  defini¬ 
tion,  does  nothing  to  measurably  alter  the  value  of  the 
forward  probabilities  (i.e.,  /rl[w)1| (  —  Xi^u)-  So,  for  exam¬ 
ple,  in  the  case  of  a  seed  growing  into  a  plant,  averaging 
(e~Pwd)  over  all  possible  ways  of  going  from  seed  to  plant 
would  give  us  a  quantity  related  to  the  total  probability 
of  turning  back  from  plant  to  seed.  In  contrast, 
averaging  (e~liWd)fv',<i  only  over  typical  forward  paths  will 
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yield  a  quantity  related  to  j  <3C  which  is  the 

probability  that  we  observe  the  plant  turning  back  into  a 
seed  via  a  path  that  looks  like  a  time-reversed  movie  of  a 
seed  growing  in  the  normal  way  into  a  plant.  We  can  thus 
denote  the  whole  thermodynamic  equation  more  compactly 
via 


‘  —fwd 


In 


_fwd 


*111 


—Ain  /  \  +  In 

\  Pbz  /  II, III 


‘ii->i 


‘iii^ii 


-In 


(exp  (~pWd)) 


fwd 

I-»II 


.(exp  (-PWd)) 


fwd 
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It  should  be  noted  in  passing  that  in  driven  systems  the 
value  of  the  reverse  probability  term  is  not  determined  only 
by  distance  traveled  or  barriers  traversed  because  external 
drives  generally  affect  the  probabilities  of  transition 
between  macrostates  in  both  forward  and  reverse  direc¬ 
tions.  Moreover,  since  drives  often  do  not  impact  forward 
and  reverse  transition  probabilities  between  states  in 
exactly  the  same  way  (since  they  push  in  particular 
directions  at  particular  times)  and  because  long  paths 
connecting  macrostates  are  made  up  of  many  partially 
independent  microscopic  moves,  we  expect  and 
between  macrostates  of  equal  equilibrium  free  energy  to  be 
positively,  but  imperfectly,  correlated  with  each  other  over 
the  range  of  possible  choices  of  *,  leading,  in  turn,  to  a 
range  of  possible  values  of  (e~^Wd)1^  for  different  end 
states. 

Taking  one  final  heuristic  step,  we  may  also  note  that 
—  ln(exp(—  fiW d))  is  a  cumulant  generating  function  [6] 
when  we  expand  in  /?,  and  thus,  it  may  be  written  as 


—  ln(exp  (-pWd))=P(Wd)- 


=  ik  -  <f>, 


(10) 


informative  in  the  case  of  an  ergodic  nonequilibrium  steady 
state  since  then  (exp[— A5tot])  =  1,  which  implies  that 
>k  -  <f>  =  0  for  the  dissipated  work  measured  over  any  finite 
time  r.  Clearly,  the  steady-state  dissipation  rate  can  take  on 
any  value,  in  principle,  depending  on  the  system  in  question. 
However,  in  such  cases,  the  dispersion  about  the  mean  rate 
of  dissipation  is  required  by  time  reversibility  to  rise  as  well 
so  that  ik  =  <b.  What  this  points  to  is  that  the  appearance 
of  <b  in  our  equation  for  relative  probabilities,  among  other 
things,  forces  us  to  ignore  contributions  to  the  total  average 
dissipated  work  that  are  associated  with  futile  cyclical 
motion  performed  while  the  system  dwells  near  a  steady 
state.  This  factor  is  therefore  closely  related  to  the  idea  of 
“housekeeping  heat”  considered  previously  in  studies  of 
transitions  between  nonequilibrium  steady  states  [8]. 

By  incorporating  this  quantity  into  our  thermodynamic 
expression,  we  may  now  write 
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Equation  (11)  has  a  rich  structure  and  may  be  thought  of  as 
a  generalization  of  the  Helmholtz  free  energy  for  the 
stochastic  evolution  of  arbitrarily  driven  nonequilibrium 
macrostates  over  finite  time.  Like  many  far-from- 
equilibrium  thermodynamic  relations,  it  essentially  just 
summarizes  the  combined  constraints  on  dynamics  that 
come  from  time  reversibility  and  conservation  of  energy 
and  probability.  What  we  would  ultimately  like  to  suggest 
here  is  that  expressing  these  constraints  in  this  form  makes 
it  possible  to  give  them  a  new  interpretation,  in  which  a 
thermodynamic  mechanism  for  nonequilibrium  self¬ 
organization  is  revealed.  In  the  next  section,  however, 
we  first  try  to  illustrate  the  relevance  of  the  above 
expression  in  a  few  simple  and  instructive  cases. 


where  'k  =  fi(  W  d)  is  the  average  dissipated  work  and  <b  is 
accordingly  defined  to  account  for  all  of  the  fluctuations 
about  this  average  via  <k  =  ln(exp(—  0Wd))  +  'k.  Because 
_  (ex y(~pw d))  >  exp {~P(Wd))  —  e_w,  it  immedi¬ 
ately  follows  that  <b  >  0,  with  equality  only  holding  when 
Wd  is  (5-function  distributed.  Thus,  <k  can  be  thought  of  as  a 
correction  due  to  the  dispersion  of  the  dissipated  work 
distribution  about  the  average  'b  that  gives  the  heaviest 
weight  to  the  leftward  tail  of  the  work  distribution.  This 
asymmetry  of  weighting  arises  because  e<]'  —  (exp[—  f)(Wd  — 
(W d)))  places  exponentially  stronger  weight  on  lower  values 
of  Wd  and  thus  grows  even  when  the  distribution  spreads 
symmetrically  in  both  directions  [18]. 

For  a  simple  example,  one  may  consider  the  case  where 
p(Wd  -  (Wd))dWd  =  (1/w)  on  the  interval  [-w/2,  w/2], 
with  w  >  0.  In  this  instance,  <b  =  In  [(2 /fiw)  sinh(/?w/2)j, 
which  grows  without  bound  in  the  limit  of  large  w  :»  kBT. 
We  can  also  note  that  the  behavior  of  <k  is  particularly 


IV.  DISSIPATIVE  HISTORY  AND 
OUTCOME-LIKELIHOOD:  TWO 
EXAMPLES 

The  nonequilibrium  systems  described,  in  principle,  by 
Eq.  (11)  are  extremely  diverse  in  their  properties,  and 
before  discussing  the  general  implications  of  the  result,  it 
will  be  helpful  to  consider  how  things  play  out  in  a  few 
simple  examples. 

A.  Driven  hopping  particle 

First,  it  is  appropriate  to  return  to  the  hopping  particle  in 
a  driven  energy  landscape  considered  in  Fig.  1.  Previously, 
we  pointed  out  that  there  was  a  direct  correlation  in  this 
system  between  the  relative  likelihood  of  two  outcomes  of 
the  driven  dynamics  and  the  amount  of  heat  dissipated  on 
the  way  to  each  destination. 
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What  we  can  now  appreciate  is  that  this  correlation 
results  from  the  weight  of  the  A\D  term  in  Eq.  (1 1).  Indeed, 
the  reason  this  effect  is  so  complete  in  this  case  is  that  our 
choice  of  external  drive  in  fact  contrived  the  two  outcomes 
of  X\  and  x3  to  have  equal  return  probability:  By  setting  the 
energy  £,  (t)  to  oscillate  in  sync  with  Bl2(t),  the  barrier 
height  for  the  return  jump  remained  the  same  as  B2 3  —  £3  at 
all  times.  Moreover,  in  the  assumed  limit  of  transitions 
dominated  by  the  lowest  instantaneous  energy  barrier,  the 
amount  of  dissipation  for  a  jump  from  x2  to  x1  is  always 
the  same  (<b  =  0)  and  always  positive  because  £j  <  E2  at 
the  moments  when  r2_^ x  (f)  is  maximized.  Referring  back  to 
Eq.  (11)  and  noting  that  AT>  =  0  in  this  case,  we  thus  can 
see  we  have  recovered  the  predicted  relationship 

~max 

-2^2  —  exp  [AT'].  (12) 

r2->3 

Using  similar  reasoning,  we  may  also  consider  the  effect 
of  the  fluctuations  (I>  on  drift.  The  above  discussion  has 
amounted  to  an  argument  for  why  scenarios  involving  drift 
in  a  particular  direction  must  also  exhibit  elevated  external 
dissipation  (that  is,  more  positive  'I'),  yet  it  is  clearly  the 
case  that  not  all  heat  evolution  in  driven  systems  is 
accompanied  by  such  statistically  irreversible  flow  of 
probability  density.  Indeed,  using  the  Crooks  relation  for 
microstate-to-microstate  transitions  in  such  a  discrete 
model,  one  may  easily  show  that  when  exp[— /?A  Q]  is 
averaged  over  all  stochastic  trajectories  that  both  start  and 
end  in  the  same  state  xh  it  is  generally  the  case  that 
=  T >  0,  reminding  us  that  much  of  the  dissipa¬ 
tion  generated  in  a  driven  system  can  result  from  futile 
cycles  that  go  around  in  circles  rather  than  drifting  in  some 
new  direction  in  phase  space  [8].  Thus,  some  ways  of 
making  'I'  more  positive  also  bring  about  an  increase  of  $ 
in  lockstep  that  cancels  any  contribution  to  the  flow  of 
probability  in  Eq.  (11).  Put  another  way,  high  fluctuations 
in  entropy  production  lessen  the  degree  to  which  drift  and 
dissipation  are  correlated. 

To  see  this  effect  of  fluctuations  in  the  case  of  our 
hopping  particle,  we  can  specialize  to  a  slightly  different 
scenario  (Fig.  4).  Now  considering  a  pair  of  states  x\  and 
x2,  we  can  initialize  the  particle  in  state  x2  with  constant 
energy  £2  =  0  and  suppose  there  are  two  distinct  paths  that 
connect  the  states.  The  first  path  has  a  barrier  height 
Bl2{t)  =  A£  +  A£cos(cof)/2,  whereas  the  second  path 
has  a  barrier  of  height  B*n{t)  =  A  £  —  A£cos(ot>f)/2,  and 
both  barriers  are  governed  by  the  same  time  constant  r.  The 
energy  of  state  x, ,  meanwhile,  is  assumed  to  vary  with  time 
as  £1  (?)  =  A£cos(cof)/2. 

In  a  system  thus  described,  we  can  straightforwardly 
compute  all  of  the  relevant  quantities  if  we  assume  we 
are  in  a  regime  where  y?A  £  1  and  r«l/r«o>  so 

that  transitions  are  rare  events  that  take  place  during 
the  moments  in  the  drive  cycle  of  their  maximum 


FIG.  4.  In  the  driven  scenario  depicted  schematically  here,  two 
states  X\  and  x2  are  separated  by  two  different  barriers  represent¬ 
ing  two  different  possible  paths  for  transiting  between  the  states. 
Because  the  barriers  are  driven  out  of  phase  with  each  other,  and 
since  /?A E  »  1  and  r  1  /t  <SC  a>,  the  return  probability  and 
average  dissipation  for  transitions  from  x2  to  x1  are  independent 
of  A E,  the  choice  of  which  should  still  affect  the  drift  rate  from  x2 
to  Xj  as  well  as  the  fluctuations  4>  for  the  pair  of  paths.  As  in 
Fig.  1,  the  energies  of  the  driven  states  are  drawn  at  their  extremal 
values  in  blue,  with  dashed  lines  and  solid  lines,  respectively,  in 
phase  with  each  other. 

likelihood.  In  this  case,  it  is  easy  to  see  that  the  particle 
is  equally  likely  to  first  jump  to  x,  via  either  banner 
since  max(r2^i(f))  =  max(r^j(f))  =  r  exp[— f)AE/2], 
Interestingly,  however,  the  mean  dissipated  work  averaged 
over  the  weights  of  these  paths  should  be  = 
P(AE/2  —  A£/2)/2  =  0,  due  to  the  fact  that  events  on 
one  path  happen  in  phase  with  the  rise  of  E1  and  events  on 
the  other  path  happen  n  radians  out  of  phase  with  it. 
Moreover,  in  the  limit  of  A  £  we  have  chosen,  the  total 
return  probability  r r  =  const  is  effectively  insensitive  to  the 
exact  value  of  the  barrier  height  since  the  return  journey 
will  always  be  made  via  B*1  and  not  B2l.  However,  in  this 
same  regime,  changes  in  A£  strongly  affect  both  the 
forward  transition  rate  and  the  fluctuations  in  entropy 
production;  specifically,  the  transition  rate  is  lnr^j  = 
ln[2re-/3A£/0(/?A£/2)]  —  —  /?A£/2,  and  the  fluctuations 
are  <h  =  0—  (— ln(e_AS))=lncosh[/)A£/2]  —  f)AE/2.  Thus, 
for  this  particular  case,  we  may  write  what  we  should  have 
expected  from  Eq.  (11): 

ln^>— (13) 

from  which  it  is  apparent  that,  as  fluctuations  in  entropy 
production  rise  with  return  probability  and  average  dis¬ 
sipation  already  held  fixed,  the  forward  probability  rate 
must  fall  sharply.  The  origin  of  this  effect  lies  in  the 
diversity  of  paths  connecting  states,  as  well  as  the  resulting 
possibility  that  the  drive  can  sometimes  help  one  return  to 
the  starting  point  via  a  different  path  than  it  pushed  one 
along  when  the  journey  first  began. 

B.  Competing  self-replicators 

In  order  to  get  a  more  phenomenological  perspective  on 
how  dissipative  history  can  be  predictive  of  outcomes  in 
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nonequilibrium  dynamics,  we  can  now  consider  two 
populations  of  self-replicating  particles,  A  and  B.  The 
coarse-grained  states  of  interest  correspond  to  different 
ordered  pairs  (NA,NB)  which  count  the  numbers  of 
individuals  of  each  type  in  the  system.  In  any  infinitesimal 
interval  of  duration  dt,  an  individual  of  type  A  or  B  can 
spontaneously  and  instantaneously  divide  into  two  indi¬ 
viduals  via  a  highly  irreversible  process  X  -*  X  +  X  with  a 
probability  of  gAdt  or  gBdt,  respectively.  Without  loss  of 
generality,  we  will  set  gB  >  gA  so  that  one  of  these 
exponentially  growing  populations  is  expected,  on  average, 
to  become  an  ever-increasing  fraction  of  the  total 
population. 

We  further  assume  that  both  types  of  self-replicators  are 
equally  likely  to  undergo  a  reversal  reaction  X  +  X  ->  X 
with  the  same  stochastic  rate  constant  5.  Although  the 
probability  for  this  process  to  happen  in  reverse  is  assumed 
to  be  very  small,  the  time  reversibility  of  the  underlying 
physics  forbids  it  from  being  zero.  Under  these  rules,  the 
probability  of  finding  N  individuals  of  type  X  in  a  given 
population  at  time  t  obeys  the  master  equation 


PNx(t)  =  —  (  gNx  +  5Nx<yN* — -  )pNx{t) 


g(Nx  -  1  )pNx-i(t) 


1  )NX 


Pnx+ i(0- 


(14) 


We  consider  the  limit  where  8  0,  but  5  must  remain 

nonzero  in  order  for  the  entropy  production 


A  Sx  =  -  In 


p[Nx^Nx-l] 

p[Nx-l^Nx] 


-In  Nx 


(15) 


dynamical  behavior  as  a  whole  from  the  way  entropy 
production  depends  on  NA  and  N B. 

Since  8  is  negligible  for  the  puiposes  of  solving  Eq.  (14) 
in  the  regime  we  are  considering,  the  probability  for 
stochastic  evolution  forward  in  time  for  both  species  from 
a  starting  condition  of  (NA,NB)  =  (1, 1)  over  time  r  is 
given  by 

p[(  1, 1)  -»•  {Na,Nb),t}  =  e~9A%{\ - 

xc-9«'(l-e-3«fr(  (16) 

which  may  be  shown  by  inspection  to  satisfy  the  master 
equation  governing  the  whole  population  in  the  <5  ->  0 
limit.  Our  goal,  however,  is  to  compute  the  reversal 
probabilities  for  returning  from  a  given  final  state  with 
population  numbers  (NA,NB)  to  an  initial  state  with  one 
individual  of  each  species,  in  a  fixed  time  interval  of  equal 
duration  z.  The  most  straightforward  way  of  doing  this  in 
this  particular  system  is  to  leave  all  the  different  contribu¬ 
tions  to  entropy  together  in  one  bundle  calculated  from 
Eq.  (15)  and  exploit  the  Crooks-type  relation  in  Eq.  (7)  that 
relates  entropy  production  to  forward  and  reverse  proba¬ 
bilities.  In  this  model,  the  total  entropy  production  caused 
by  a  population  as  it  goes  from  one  individual  of  each 
species  to  NA  and  NB  of  them,  respectively,  will  be 
assumed  as 


,  />[(!,!) ->(#a,1Vb),t] 
np[(NA,NB)  -+  (1,1),  t] 

—  AS[(1>  1)  -*■  (Na,Nb)] 


=  E 

X={A,B} 


(Nx-\)\n 


2ffx 

8 


-In  Nx\ 


(17) 


associated  with  each  replication  event  to  remain  finite  [14] 
(Note:  The  form  of  the  entropy  production  expression  takes 
a  slightly  different  form  here  than  in  Ref.  [14]  because  the 
reverse  path  defined  here  occurs  by  pair  annihilation,  rather 
than  single-particle  decay).  We  therefore  restrict  our  atten¬ 
tion  to  the  case  where  8  is  nonzero  but  exponentially  small 
compared  with  the  growth  rate:  \n(gx/[8N x\)  1. 

Our  goal  here  is  to  illustrate  that  external  entropy 
production  becomes  predictive  of  the  stochastic  dynamics 
of  this  system  once  comparisons  are  made  between 
possible  outcome  states  that  all  have  the  same  value  (or 
greater)  of  spontaneous  reversal  probability.  It  must  be 
emphasized  that  at  no  point  is  it  assumed  here  that  this 
reversal  probability  is  constant  across  all  possible  choices 
of  ordered  pairs  (NA,NB).  On  the  contrary,  this  quantity  is 
expected  to  vary  dramatically  depending  on  how  NA  and 
Nb  are  chosen.  As  we  shall  see,  however,  the  examination 
of  a  large  set  of  such  ordered  pairs  that  share  the  same  value 
of  this  reversal  term  will  allow  us  to  make  a  correct 
inference  about  an  emergent  quality  in  the  system’s 


which  accounts  both  for  the  internal  entropy  of  the  mixture 
of  indistinguishable  self-replicators  and  for  the  heat  dis¬ 
sipation  in  the  surrounding  bath.  Combining  this  expres¬ 
sion  with  the  one  for  p [( 1 , 1 )  -*■  (NA,  NB),z]  enables  us  to 
write  the  reversal  probability  in  time  r  as 


p[(Na,Nb)  -  (1,1),  t] 

—  e~9AT(  \  —  e~9AT)NA~l 

/  s  \  na-i 

x  e~9BT(l  -  e~9riT)Nl‘~i  - 

V2W 

(  8  \N"-' 

*  U  w- 


(18) 


Although  we  have  obtained  this  result  by  assuming 
thermodynamic  consistency,  one  may  confirm  that  it  is 
correct  by  computing  the  quantity  directly  as  a  path  integral 
over  individual  microtrajectories. 

All  ordered  pairs  ( NA  >  0,  NB  >  0)  are,  in  principle, 
possible  outcomes  of  this  process  after  a  time  r,  and  under 
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Na  xlO8 

FIG.  5.  Lines  of  fixed  reversal  probability  based  on  Eq.  (18), 
with  gA  =  1,  gB  =  2,  z  =  10,  and  5  =  e~wo.  Each  line  is  colored 
to  show  the  entropy  produced  while  generating  each  ending  state 
from  the  initial  state  NA  =  NB  =  1,  relative  to  that  of  the  state  on 
the  line  where  NB  =  NA.  On  each  of  these  surfaces  of  fixed 
reversal  probability,  the  most  likely  outcome  by  orders  of 
magnitude  is  one  where  NB  »  NA. 

the  limits  we  have  described,  their  associated  return 
probabilities  vary  by  many  orders  of  magnitude  (Fig.  5). 
There  are,  however,  lines  through  this  state  space  on  which 
the  reversal  probabilities  are  constant,  and  we  can  use 
Eq.  (18)  to  obtain  the  relation  between  NA  and  NB  that 
defines  these  lines.  We  are  interested  in  this  relation  in  the 
limit  of  vanishingly  small  5  (In (gx/[8Nx\)  1),  Nx  1 
and  comparable  order-of-magnitude  sizes  for  gA  ~ 
9b  ~  (1/r)  In  Nx.  In  this  case,  by  taking  the  logarithm  of 
Eq.  (18)  and  setting  it  equal  to  a  constant,  C,  we  find  that 
the  dominant  contribution  comes  from  the  Nx  In  <5  terms, 
and  we  can  write  N  =  NA  +  NB  —  C/  In  <5,  so  fixed-rever¬ 
sal  probability  lines  nearly  become  lines  of  fixed  total 
population  number.  In  fact,  this  correspondence  cannot  be 
exact  since  the  discreteness  of  integer  lattice  sites  prevents 
most  states  from  having  identical  values  of  reversal 
probability.  We  can,  however,  use  the  collection  of  lattice 
sites  with  fixed  particle  number  N  as  a  proxy  for  the  fixed 
return  probability  ensemble  in  its  role  as  a  large  and  diverse 
set  of  outcomes.  On  such  level  sets,  the  entropy  production 
takes  the  simple  form 

AS  =  C'  +  In p[(l,  1)  {Na,Nb),t\ 

»  C"  +  NB{e-^T  -  e-w),  (19) 

where  C' ,  C"  are  constants  independent  of  NA  and  NB. 

Figure  5  shows  how  AS  spans  a  wide  range  of  values 
over  each  line  of  fixed  reversal  probability  for  a  typical 
choice  of  parameters.  Since  gB  >  gA,  the  expression  in 


parentheses  is  always  positive,  so  entropy  production 
increases  with  increasing  NB  along  each  line.  It  further 
follows  that  the  relative  likelihood  of  states  on  this  level 
curve  increases  exponentially  with  NB,  so  we  obtain  here 
the  result  for  this  model  that  in  any  half-plane  bounded 
from  above  by  some  value  of  reversal  probability,  the  most 
likely  outcomes  (which  are  also  those  that  have  produced 
the  most  entropy)  are  those  where  NB  5s>  NA. 

It  is  worth  restating  the  flow  of  the  above  argument.  Of 
course,  we  already  find  it  obvious  that  in  a  competition 
between  two  exponentially  growing  populations  of  indi¬ 
viduals  that  essentially  never  die,  the  type  of  individual  that 
grows  at  a  faster  rate  is  expected  to  become  more  numerous 
with  the  passage  of  time.  This  intuition  is  all  we  need  to 
understand  the  dynamics  of  this  particular  system. 
However,  our  goal  here  is  to  provide  an  explanation  for 
this  outcome  in  a  thermodynamic  language  that  will  remain 
applicable  in  settings  where  the  stochastic  dynamics  are 
much  less  straightforward  to  predict  by  other  means.  Here, 
total  entropy  production  was  determined  simply  by  the 
number  of  self-replication  events  of  each  type,  whereas 
reversal  probability  essentially  tracks  the  total  population 
size.  Once  we  bound  this  latter  quantity  from  above,  we 
find  that  the  likeliest  outcomes  (which  are  those  for  which 
Nb^$>  Na)  always  turn  out  to  be  those  for  which  A 5  is  most 
positive,  given  the  reversal  constraint. 

Conversely,  we  can  say  that  if  we  had  no  a  priori 
intuition  for  how  the  system  should  behave,  we  could  still 
infer  the  likely  takeover  by  species  B  from  the  fact  that  A 5 
increases  with  NB  along  lines  of  fixed  reversal  probability. 
Here  the  “structural  quality”  that  correlates  strongly  with  an 
exceptional  dissipative  history  on  any  fixed  reversal  prob¬ 
ability  surface  is  having  NB  NA,  and  it  is  this  quality  that 
indeed  is  expressed  in  the  outcome  of  the  stochastic 
dynamics.  Moreover,  if  we  instead  looked  at  surfaces  of 
fixed  entropy  production,  we  would  uncover  an  inverse 
relationship  between  reversal  probability  and  total  particle 
number.  Accordingly,  although  an  infinite  number  of 
different  particle  numbers  are  possible,  in  principle,  at 
any  finite  time,  we  expect  (and  find)  that  sufficiently  large 
values  of  N  are  less  likely  to  be  observed  than  smaller  ones. 

The  examples  we  considered  here  effectively  illustrate 
why  time  reversibility  in  thermally  fluctuating  systems 
implies  a  general  relationship  between  the  concerted  flow 
of  probability  density  and  the  consistent  conversion  of 
work  to  positive  amounts  of  dissipated  heat.  In  the 
remainder  of  this  article,  we  discuss  how  this  relationship 
may  be  construed  to  suggest  a  thermodynamic  mechanism 
for  adaptive  self-organization. 

V.  DISCUSSION 

In  this  article,  we  have  demonstrated  a  general  expres¬ 
sion  for  the  time  evolution  of  the  probability  distribution 
over  macrostates  for  a  driven  classical  many-body  system. 
Moreover,  we  interpreted  this  equation  to  be  a  summary  of 
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the  distinct  pressures  pushing  the  system  towards  particular 
outcomes.  The  first  such  pressure  is  the  most  obvious  one: 
All  things  being  equal,  we  expect  our  collection  of  matter  in 
contact  with  a  heat  bath  to  tend  towards  thermal  equilib¬ 
rium.  Next,  we  account  for  kinetic  accessibility:  Not  all 
arrangements  of  a  system  are  equally  “adjacent”  or  “close” 
to  a  given  starting  point,  and  thus,  only  certain  arrange¬ 
ments  should  be  considered  equally  accessible  on  any  finite 
time  scale  of  evolution.  Beyond  these  effects,  the  rest  of 
what  determines  relative  likelihood  is  a  function  of  a  state’s 
implicit  history  of  thermodynamic  flux.  Outcomes  from 
absorption  and  dissipation  of  work  that  is  consistently  high 
(corresponding  to  exceptionally  positive  'I'  and  small  <f>)  are 
expected  to  be  favored  once  the  kinetic  and  equilibration 
terms  are  held  fixed  in  an  all-things-equal  comparison. 

It  was  easiest  to  see  how  such  a  general  thermodynamic 
statement  plays  out  in  the  two  simple  examples  we  already 
considered,  but  now  we  may  ask  whether  the  same 
theoretical  framework  might  reveal  interesting  conse¬ 
quences  in  more  complex  nonequilibrium  settings.  To 
answer  this  question,  we  pose  a  thought  experiment  that 
will  allow  us  to  see  implications  for  systems  that  seem  to 
undergo  qualitative  transformations  in  their  pattern  of 
organization  as  a  result  of  strong  external  driving. 

Imagine  an  experimental  realization  of  a  process  of 
stochastic  evolution  for  a  many-particle  system  in  which 
some  particular  outcome  called  II  is  reliably  the  one 
observed  after  the  system  is  initialized  in  some  nonover¬ 
lapping  state  I  and  driven  for  finite  time  r  with  some 
particular  pattern  of  time  variation  in  the  external  field  2(f). 
For  a  concrete  example,  one  might  consider  the  phenome¬ 
non  of  shear  thickening  in  a  driven  colloid,  where  particle 
clumping  is  caused  by  the  application  of  external  shear  to  a 
formerly  dispersed  mixture  [19,20].  By  requiring  that 
2(0)  =  2(t)  and  that  F\ [q  <  Fjj,  we  avoid  the  trivial  cases 
where  the  transition  in  question  would  be  favored  by 
undriven  dynamics  of  equilibration.  Moreover,  we  assume 
that,  while  the  forward  probability  of  evolution  7rj^n  is 
close  to  1,  the  reverse  probability  1  is  very  small 

(which,  in  turn,  usefully  implies  that  ^jejv-»i  <  nu^i  <SC  1  is 
also  very  small),  so  the  effect  of  the  drive  is  not  simply  to 
symmetrically  accelerate  the  kinetics  of  exchange  between 
I  and  II.  Lastly,  we  need  to  posit  that,  with  the  driving 
field  turned  off,  the  forward  transition  probability 
should  be  much  less  than  what  results  in  the  presence  of  the 
drive.  Taken  together,  these  stipulations  mean  that  we  are 
aiming  specifically  to  consider  a  system  where  introducing 
the  time-varying  external  driving  field  triggers  a  reliable 
and  irreversible  configurational  shift  into  a  state  that  would 
not  normally  be  explored  as  a  result  of  thermal  fluctuations 
alone  in  the  same  amount  of  time. 

In  the  scenario  thus  described,  the  assumptions  we  have 
already  stated  are  sufficient  to  imply  that  II  will  have  the 
optimally  positive  value  of 'T  —  across  a  large  and  diverse 
set  of  possible  macroscopic  arrangements  for  the  system. 


Of  course,  not  all  arrangements  of  a  many-body  system 
will  be  equally  kinetically  trapped;  thus,  not  all  macrostates 
we  construct  will  satisfy  our  assumption  of  < 

k n^i  <s:  1.  However,  in  a  system  containing  many  differ¬ 
ent  interacting  degrees  of  freedom,  the  space  of  possible 
particle  arrangements  that  do  satisfy  this  inequality  should 
be  expected  to  remain  combinatorially  vast,  and  it  is  over 
these  possible  outcomes  that  the  overwhelmingly  likeliest 
one  (II)  must  have  an  exceptional  implicit  history  of 
consistently  positive  dissipative  flux.  (Note:  Both  the  focus 
on  dissipative  history  and  account  of  the  effect  of  fluctua¬ 
tions  through  the  contribution  of  the  <I>  term  serve  to 
distinguish  the  result  discussed  here  from  maximum 
entropy  production  principles  that  have  been  explored 
elsewhere  [21].) 

What  does  having  an  exceptionally  dissipative  past  mean 
for  the  way  the  system  is  configured  in  the  present?  To 
provide  an  answer,  it  is  essential  to  realize  that  not  every 
microscopic  arrangement  x  available  to  our  system  of  interest 
will  absorb  the  same  amount  of  work  from  the  same  time- 
varying  field;  indeed,  the  instantaneous  work  rate  of  the  field 
at  any  given  moment  is  W  —  dHsys(x,A(t))/dt,  which 
generally  must  depend  on  the  value  of  x  [6,7].  This  point 
can  be  made  most  intuitive  by  considering  a  system  with  a 
rugged  energy  landscape  characterized  by  the  existence  of 
many  degenerate  local  minima  that  might  act  as  kinetic  traps 
below  a  certain  temperature.  Small  deviations  from  one  such 
local  minimum  should  behave  harmonically  and  give  rise  to  a 
spectrum  of  corresponding  normal-mode  frequencies  specific 
to  the  local  steepness  of  the  potential.  Consequently,  when 
assessing  how  much  work  will  be  absorbed  by  the  system 
when  it  is,  for  example,  sinusoidally  driven  by  an  external 
force  with  frequency  co,  a  pertinent  question  will  be  the 
following:  How  many  normal  modes  for  the  system  have 
natural  frequencies  close  to  m,  such  that  they  might  exhibit  a 
high-amplitude,  high-dissipation  response  to  the  drive?  This 
number  of  such  modes  could  vary  substantially  across  the 
vast  space  of  many-particle  arrangements.  Thus,  from  such 
an  example,  we  can  take  the  more  generic  intuition  that 
special  subsets  of  particle  configurations  may  be  particularly 
well  (or  poorly)  suited  to  absorb  work  from  a  given  drive. 

We  expect,  then,  that  in  many  cases  where  external 
driving  brings  about  a  strong  shift  in  a  system’s  state  of 
organization,  the  outcome  we  observe  at  long  times  should 
reflect  that  the  system  had  to  be  specially  configured  at 
some  time  in  its  past  in  order  to  absorb  and  dissipate  large 
amounts  of  work  from  the  drive.  Of  course,  exactly  how 
such  a  tendency  toward  “dissipative  adaptation”  to  the  drive 
plays  out  could  lead  in  very  different  directions  depending 
on  the  details  of  the  system.  For  example,  in  a  scenario 
where  absorption  and  dissipation  of  work  from  a  drive  fuels 
a  process  of  positive  feedback  (such  as  in  self-replication), 
one  expects  the  system  to  exhibit  an  improving  ability  to 
cause  dissipation  in  its  surroundings  over  time.  On  the  other 
hand,  in  scenarios  where  the  absorption  and  dissipation  of 
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work  causes  highly  irreversible  structural  changes  that 
cause  a  drop  in  the  rate  of  work  absorption  (such  as  when 
resonant  vibrations  with  a  song  cause  the  shattering  of  a 
wine  glass  into  shards  that  no  longer  resonate  well),  one 
could  observe  an  end  state  with  a  lessened  ability  to  absorb 
work.  In  either  event,  however,  we  expect  that  the  eventual 
configuration  of  the  system  should  be  correlated  in  an 
informative  way  with  the  moments  in  its  history  when  it 
absorbed  large  amounts  of  work  in  order  to  undergo  highly 
irreversible  configurational  changes. 

Strikingly,  this  manner  of  thinking  can  be  thought  of  as 
the  natural  generalization  of  the  way  people  often  reason 
about  evolutionary  adaptations  in  Darwinian  examples  in 
biology:  When  a  self-replicator  exhibits  adaptive  traits  that 
match  well  with  its  present  environment,  we  can  seek  an 
explanation  in  the  hypothesis  that  the  organism’s  ancestors 
possessed  these  traits  and  found  them  helpful  in  the 
struggle  to  survive  and  generate  many  offspring  [22].  As 
we  have  already  seen  in  the  previous  section,  the  link 
between  this  picture  and  the  thermodynamic  one  comes 
from  the  fact  that,  as  a  result  of  its  statistical  irreversibility, 
self-replication  is  necessarily  a  dissipative  process  [14],  so 
exponential  population  growth  turns  out  to  be  one  very 
reliable  mechanism  of  driving  external  entropy  production. 
In  this  more  general  physical  frame,  however,  we  can 
appreciate  that  even  a  system  devoid  of  any  recognizable 
self-replicator  might  still  end  up  seeming  “well  adapted”  to 
environmental  drives  in  light  of  the  marks  on  its  current 
configuration  left  by  its  exceptional  dissipative  history. 

VI.  CONCLUSION 

It  has  certainly  long  been  known  that  an  endless  variety 
of  far-from-equilibrium  many-particle  Newtonian  systems 
are  capable  of  exhibiting  self-organization  phenomena  in 
which  strikingly  patterned  structures  emerge  in  the  pres¬ 
ence  of  dissipative  external  drives  [5].  Whether  in  sand 
dunes  or  snowflakes,  in  hurricanes  or  in  spiral  bundles  of 
protein  filaments  and  motors  [3,4],  the  nonequilibrium 
world  offers  many  test  cases  for  the  general  hypothesis  that 
organized,  kinetically  stable  structures  emerge  and  persist 
because  their  formation  is  accompanied  by  extra  work 
absorption  and  dissipation  [1,2].  While  the  story  in  each  of 
these  cases  must  be  different  in  many  of  its  details,  we  may 
speculate  that  a  thermodynamic  commonality  would  be 
revealed  to  underlie  all  of  them  if  the  right  physical 
observables  were  tracked  and  compared. 

Guided  by  these  theoretical  considerations,  we  recently 
demonstrated  the  predicted  phenomenon  in  a  simple 
simulation  framework  that  tracks  the  vibration  spectrum 
of  a  sinusoidally  driven  toy  chemical  mixture  over  time. 
The  main  result  of  the  study  (which  will  be  published  in  a 
separate  article,  rather  than  shown  here)  is  that  we  do 
indeed  see  emergent  “adaptive”  resonance  in  the  system, 
such  that  our  choice  of  frequency  for  an  external  driving 
field  determines  the  location  of  the  peak  in  the  resonance 


spectrum  for  a  particle  mixture  that  evolves  stochastically 
in  the  presence  of  such  an  environmental  drive.  This  finding 
turns  out  to  be  highly  suggestive  of  the  results  of  recent 
experiments  performed  on  silver  nanorods  assembling  in 
the  presence  of  light  fields  of  different  colors  [23],  which 
similarly  “learned”  to  match  their  surface  plasmon  reso¬ 
nance  to  the  frequency  of  the  driving  field.  Thus,  we  are 
encouraged  to  explore  further  with  related  models,  spurred 
on  by  the  intriguing  possibility  that  lifelike  adaptive 
behavior  in  nonequilibrium  systems  may  turn  out  to  be 
surprisingly  common,  now  that  we  have  begun  learning 
how  to  look  for  its  physical  signatures. 
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